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Abstract 

This design research type development study aims at producing a set ofPISA-like 
mathematics task which is valid and practical as well as has the potential effects. Then, 
activation of fundamental mathematical capabilities underlying mathematical process related 
to the mathematical literacy as the main potential effect of the developed PISA-like tasks 
would be the focus on this paper. The subject of this study were 28 seven graders of SMP 
Negeri 1 Palembang involved in field test. Data collecting techniques used were students’ test 
result and interviews. Overall findings indicate that 10 items of PISA-like mathematics task 
developed potentially promote students’ mathematical literacy within three mathematical 
processes. The result also shows that the highest percentage of students’achievement was on 
the interpreting tasks (52.55%). While achievement for employ and formulate tasks were 
40.74% and 39.63% respectively. 

Keywords: PISA-like problems, mathematical literacy, mathematical process, fundamental 
mathematical capabilites 


Abstrak 

Penelitian desain riset tipe pengembangan ini bertujuan untuk menghasilkan seperangkat soal 
matematika model PISA yang valid dan praktis. Selanjutnya, aktivasi kemampuan dasar 
matematika dalam proses literasi matematis sebagai efek potensial utama dari soal yang 
dikembangkan merupakan fokus utama dari paper ini. Subjek penelitian adalah 28 siswa kelas 
VII SMP Negeri 1 Palembang yang terlibat dalam field test. Data yang dikumpulkan berupa 
hasil tes dan wawancara. Hasil penelitian secara umum mengindikasikan 10 butir soal model 
PISA yang dikembangkan berpotensi mampu meningkatkan kemampuan literasi matematis 
siswa melalui 3 proses matematika. Selanjutnya, rata-rata pencapaian siswa tertinggi terjadi 
pada soal dengan kategori menafsirkan (39,63%). Sedangkan untuk dua kategori lainnya, 
menerapkan dan merumuskan adalah sebanyak 40,74% and 52,55% secara berturut-turut. 

Kata kunci: soal model PISA, melek matematika, proses matematis, kemampuan dasar 
Matematis 


Programme for International Student Assesment (PISA) is a large-scale assessment conducted every 
three years which is coordinated by Organization for Economic Co-operation and 
Development/OECD (OECD, 2013).This international study is regularly administered to examine how 
well 15-years-old students are preparedat school to apply their knowledge, including mathematics, to 
analyse and solve problems in a variety of real-world situations (OECD, 2013; Stacey, 2011). 

Mathematical literacy, one of focuses of PISA study, refers to an individual’s capability to 
formulate, employ, and interpret mathematics in a variety of real-life situations which assists 
individuals to recognise roles that mathematics plays in the real world (OECD, 2013) and prepares 
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them for their future lives and work (Gatabi, Stacey, & Gooya, 2012). In other words, it is an 
essential ability that every mathematics learners should grasp. Therefore, it is highly expected that 
mathematics learning process could potentially promote students’ mathematical literacy (Gatabi, et al., 

2012) . All five-PISA surveys’ results, nevertheless, steadily indicate Indonesian students’ low 
performance related to the mathematical literacy. Even, PISA 2012 the newest survey revealed that 
75.7% of Indonesian students were merely able to work up to level 2and around 0.3% of the students 
could only answer mathematics tasks embedded in complex situations on top-two levels (OECD, 

2013) . 

Related to these survey results, definitely, many factors might have caused the low of 
Indonesian students' mathematical literacy; e.g. students are not accustomed to solving PISA-like 
tasks in their learning activity (Ahyan, Zulkardi, & Darmawijoyo, 2014) since they were simply used 
to obtaining and using formal mathematics knowledge in the classroom (Novita, Zulkardi, & Hartono, 
2012) as well as the lack of students’ problem solving skills for non-routine or high level problems 
since most of the test items used in the learning process is only focus on the low level tasks (Novita, et 
al., 2012) and identified to be unsufficient to promote students’ ability in solving contextual problems, 
including PISA-like mathematics tasks (Wijaya, 2015). Kemendikbud (2013) also supported these 
issues that a lot of test material/ questions administered in PISA study was not included in Indonesian 
mathematics curriculum. 

To deal with these problems, Indonesia has implemented the newest curriculum what so called 
curriculum 2013 in which one of its development background was encouraged by the PISA results 
(Kemdikbud, 2013). Then, as the contribution to the implementation of this curriculum, mathematics 
tasks which could support students’ mathematical literacy enhancement, such as PISA-like problems, 
is important to develop. Kohar (2013) supported this idea by claiming that the need of developing 
learning resources fostering students’ mathematical literacy is important to do. Zulkardi (2002) also 
suggested to develop PISA-like mathematics tasks as well as to use them in instructional practices. 

Considering those cases, this study attempted to develope a set of PISA-like mathematics tasks 
which are valid, practical, and have potential effects toward seven grade students. The activation of 
fundamental mathematical capabilities, or simply called as FMC, which underpins process of 
mathematical literacy as the main potential effect of the developed PISA-like tasks then would be the 
focus of this paper. 

As proposed by Turner and Adams (2012), FMC involving seven capabilities; communication, 
mathematising, representation, reasoning and argument, devising strategies for solving problems, 
using symbolic, formal and operations, and using mathematical tools underlies three mathematical 
process (formulate, employ, and interpret) as the components of mathematical literacy. Turner and 
Adams (2012) then argued that as the level of mathematical literacy possessed by an individual 
increases, that individual is able to draw to an increasing degree on the fundamental mathematical 
capabilities. 
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The central issue of this present study, therefore, is formulated into a research question: how do 
the seventh graders activate their FMC underlying process of mathematical literacy in solving PISA- 
like mathematics tasks ? 

METHOD 

This is a design research type development study including two phases; preliminary and 
prototyping with formative evaluations (Nieveen & Plomp, 2007) consisting of self evaluation, expert 
reviews and one-to-one as well as small group and field test (Tessmer, 1993; Zulkardi 2002). 



Figurel.Formative evaluation (adapted from Tessmer, 1993; Zulkardi, 2002) 


There were 6 experts from both Indonesia and overseas as well as 43 seven graders of SMPN 1 
Palembang involved in the prototyping phases; in which 5 students as the subject of one-to-one, 10 
students for small group subject, and 28 students as the field test subject. This paper, however, only 
focuses on the field test result. In this term, the result of the field test is students’ FMC activation 
within mathematical processes related to the mathematical literacy as the potential effect of developed 
PISA-like problems. Data collection techniques used were students’ test results and interview. 

In the preliminary stage, the researcher developed a set of PISA-like mathematics tasks, or 
simply called as initial prototype, and other supporting instruments e.g. rubric, interview guidance, and 
walkthrough sheet. Initial prototype was then self-evaluated which results in prototype 1. Such a 
prototype was then validated simultaneously in both expert reviews and one-to-one stages. Some 
substantial comments and suggestions acquired from those two phases were then used to emend the 
existing prototype to produce prototype 2 which was re-evaluated in a small group. Students’ 
suggestions in the small group were used to revise the prototype 2 to produce a prototype 3. Fifteen 
items of PISA-like mathematics tasks in this prototype were then used to assess student’s real 
performance in a larger test, namely field test. In this field test, students were asked to solve a set of 
PISA-like mathematics problems. After finishing 10 items for around an hour, some of them were also 
interviewed to confirm and explain their answers.In this paper, researcher concerns about students’ 
performances related to mathematical literacy by assessing their FMC activations underlying 
mathematical processesswhen solving the prototype 3 in the field test. 
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RESULTS AND DISCUSSION 

Field test attempted to investigate the potential effects of a set of PIS A-like problems which had 
been developed. In this term, potential effects could be implied from the students’ FMC activations 
underpinning mathematical process as the main component of mathematical literacy. To deal with this 
case, students’ test results and interviews in the field test were analyzed to find out such emerging 
potential effects. 

The following are two item examples assigned to students followed by involvement of students’ 
mathematical process (formulate, employ, and interpret) as well as fundamental mathematical 
capabilities related to the mathematical literacy when they solved such tasks. 


Context Unit 1: Car Garage 



Figure. 2. Mrs. Faridah and her car 
Source: Personal documentation 



Length of Garage 


Figure 3. Garage plans 

Source:http://gambardesainr umah99.com/gambar/2013/11/ 
Denah-Rumah-Idaman-Tipe-45-2014.jpg 

A bricklayer was asked to construct a car garage at Mrs. Faridah’s house. To determine its 
size, he was only given a photo of Mrs. Faridah and her car (see figure 1) as instruction as 
well as information that Mrs. Faridah’s height is 165cm. 

How does the ideal length of garage that the bricklayer should construct if this garage fits 
for a car as represented in the photo (see figure 2). (Note: garage length followed by the car 
length) 

Choose a correct answer below 

a. 3 meter 

b. 3.5 meter 

c. 5 meter 

d. 12 meter 


This ‘car garage’ item is belong to the interpreting process category. Indeed, the major cognitive 
demand in this problems is reflecting upon the results obtained in the employing process and 
examining them back into the problem whether it makes sense or not. In this term determining the 
length of an ideal garage needed by Ms. Faridah becomes the main problem of this item. 
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In terms of FMC demanded, this item highly requires representation competency. While 
reasoning and argument would be activated by understanding the relationship between information of 
the person’s height and the carin photo. Then , formulating a mathematical model of the relationship 
those two elements becomes pre-requisite of mathematising activation. 

To determine the length of the car, the strategy needed is estimating the ratio of person’s height 
and the car’s length on the photo (see figure 2). The length of the car could be estimated through 
employing simbolic/simple algebraic operation by using the provided information of the person’s 
height. This process continues to interpret the length of car and ‘ideal garage’ needed. In this case, 
understanding phrase ‘ideal garage’ as a part of receptive communication is highly essential drawing 
the final solution. Regarding the varied options proposed on the problem, students are re-highly 
considered to reason and interpret the acquired solution as well as evaluating them back into the 
context of problem whether the conclusion is reasonable or not. 

Based on the analysis of students' answer, 32.14% of students answered correctly by choosing 
option C (5 meters). Meanwhile, there were 39.29% and 28.57% students who chose option B (3.5 
meters) and A (3 meters) respectively. This result was subsequently confirmed through interviews 
with four students. Such brief interviews indicated 2 out of 4 students (A-10 andA-13) answered 
correctly, while the rest: A-01 failed in the process of formulating and A-12 failed when interpreting 
the final conclusion. 

A-12 is one of students who got no credit in this item. Although he unccorrectly answered, the 
fragment 1 shows activation some competencies FMC on his work. It indicates that he had well- 
activated representation and reasoning competencies by associating multiple relevant information of 
the pictorial representation. It could be noticed from the fragment 1 at lines 2 and 4 that A-12 was able 
to link the information about car and Mrs. Faridah as provided on the photo to find the length of the 
real car. He then could transformed the information into mathematical model. A-12 also well 
performed the mathematical operations in estimating the car length by taking the person’s height as 
the reference point, thus he could obtained 3.3 meter as the real car length. Unfortunately, A-12 failed 
to interpret the relation between the real car length and the ideal length of garage which brings him to 
the wrong conclusion. He could not interpret whether 3,5 meters, the result of employing process he 
obtained, is reasonable or not to be the length of an ideal garage. It is indicated by his argument as 
stated in the fragment 1 at line 11 and 12. Some involved competencies A-12 performed, however, 
basically proved that he had partly-succeed in activating some competencies of FMC when solving 
this task. 

The following is the fragment of interview result between researcher (R) and A-12, a student 
who got no credit on ‘car garage’ problem. 
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Fragment 1. Interview with A-12 


1 . 

R 

“What is your strategy to solve this problem?” 

2. 

A-12 

“Estimating the ratio between car and Mrs. Faridah. I got she is twice as height as 
the car” 

3. 

R 

“For what you estimated it?” 

4. 

A-12 : 

“To find the length of the car.” 

5. 

R 

“And what was the result you obtained?” 

6. 

A-12 : 

“330 cm.. 165 plus 165.. or 3,3 meters” 

7. 

R 

“An then, how about your final answer?” 

8. 

A-12 

“I opted B. 3, 5 meters. Since 3,5 is the most closed value for 330 cm” 

9. 

R 

“Are you sure?” 

10. 

A-12 

“Yes.” 

11. 

R 

“How about the rest of garage space? Is it fit for ideal garage?” 

12. 

A-12 

“(think for a moment) Yes. Enough.” 


[it Oh 

a 3 meter 

St 3,5 meter 

(9 5 " ,c ' er » (?><✓. 

d. 12 meter 

Figure 4. Student’s answer for full credit on ‘car garage’ unit 

Different idea was expressed by another student. A-10. Answer of student on figure 4 briefly 
indicates the involvement of FMC on his work. Starting from the process of formulation, he was able 
to make a relationship between a person’s height and the length of the car in the photo representation. 
Then, through a good employ process, A-10 could obtained the length or the real car, i.e. 3.3 meters by 
using simple algebraic operations. Finally, he succeed to reflect upon the solution he acquired and 
interpret it back into the context of problem whether the conclusion makes sense or not. From the 
answer document on figure 4, it is notable that at first A-10 chose 3.5 meters. After evaluating the 
result he obtained back into the problem context by comparing the rest of the land for both garage (5 
and 3.5 meters), he eventually decide that the most logical answer is 5 meters for the ideal length of a 
garage. It basically proves that he was able to well perform all demanded FMC within mathematical 
processes in solving ‘car garage’ task. 
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Context Unit 2: Fuel Price 


At November 1, 2014, Indonesia’s government had a plan to increase the fuel price 
for premium and diesel as shown on Kompas, one of Indonesia’s television channels. 


1 NOVEMBER 2014 



SEBELUMNYA 

RP 6.500 

MENJADI 

RP 9.500 


SOLAR 

SEBELUMNYA 

MENJADI 


RP 5.500 
RP 8.500 


Initial price 
Current price 

DIESEL 

Initial price 
Current price 


I UK b.5UU 
IDR 9.500 


IDR 5.500 
IDR 8.500 


Figure 5. Increasing price of premium and gasoline 
Source: https://sijidewe.files.wordpress.com/2014 


Dino said that the percentage of increasing price for both premium gasoline and diesel 
were same. 

Do you agree to what Dino argued? Explain your reasoning. 


Fuel price was picked to be one of unit contexts in this study since it was considered to be 
realistic and quite familiar in students’ daily life. To solve this problem, students need to compare the 
percentage of increasing price for two types of fuel (premium and diesel). Formulating process is then 
considerably dominant in this item because students are demanded to represent and transform a real- 
world situations such as information of a pictorial representation into a formal mathematical structure. 

In terms of FMC involved, this item requires representation to comprehend all of information 
on the pictorial representation. Reasoning and argumentation then would emerge by associating 
information about the initial and current price of two different types of fuel as well as concept of 
increasing price percentage. In this case, understanding phrase ‘percentage of increasing price’ as a 
part of receptive communication is highly essential before coming to the mathematising. For devising 
a strategy, students need to find the percentage of price increasing for each type of fuel (gasoline and 
diesel). By comparing those values, students then could draw the final conclusion that the price 
increasing percentage of the diesel is higher than the premium. A set of mathematical calculation 
performed in such process is the part of using symbols, operations and formal languages activation. 

Analysis of students’ answer in this item showed 35.71% of students were able to obtain full 
credit and 28.57% of students received partial credit. While the rest, 35.72% of students got no credit. 
It briefly indicates that most of students had activated their mathematical literacy through activation of 
FMC within mathematical process, although some of them merely able to partly-activate such 
competences. 

Document of student’s answers, A-27 and A-14, on figure 6 are two of the wrong answer 
examples in this item analysis. Although both of them notably made a wrong conclusion as the result 
of incomplete steps they performed, they showed activation some capabilities of FMC in their work. 
They could understand well information of pictorial representation. By activating representation 
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competence, they were able to translate and comprehend the meaning of ‘initial and current price’. It 
is notable in the fragment at lines 2 to 4.Then by activating reasoning ability, they could make the 
relationship between those elements before shifting to the matliematization and employing process. A- 
27 and A-14 also proposed additional reasoning and argument by considering the cheaper initial price 
of diesel than that of premium. It is indicated by A-27’s argument as stated in the fragment 2 at line 8. 


Such argument was basically showed that the reasoning abilities of student had been activated but it 


was constrained to a lack of students’ formal understanding about the concept of percentage. 


sew* -- 

P^ep)'urn* < 55 >oo- tcoo - ^ooo naiw 

i(MlAr S St/0 “ S'-SOO ' 3 Oou OcmV. 

fiMTwn pfpin>viv\ [etuV 

A-27 

Agree, 


premium: 9.500 - 6.500 = 3000 increasing 3.000 

diesel: 8.500 — 5.500 = 3.000 J 


However initial price of premium was more expensive than 

that of diesel 
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a r i'3-c»o 

/foe*, 

A-14 


I agree directly since they both have the same increasing price: 3000. However, I undirectly 
disagree with Dino’s argument since diesel price is cheaper than that of premium 


Figure 6. Students’ work (A-27 and A-14) whose partial credit 


Such findings were reinforced by interview data of researcher (R) with a student (A-27) as 
follows. 


Fragment 2. Interview with A-27 


1. R : “How did you solve this task?” 

2. A-27 : “Find the difference between initial and current price for both fuel types.” 

3. R : “How much the price increasing for both?” 

4. A-27 : “IDR 3.000 for both gasoline and diesel.” 

5. R : “How about the proportion of their increasing price percentages?” 

6. A-27 : “Maybe., same, (think over) 

7. R : “Are you sure? Although their price initially differ?” 

8. A-27 : “That is make me confuse. Thus, just now I wrote agree if considering the amount of 

increasing price. However, I undirectly disagree since the diesel’s price was 
cheaper than that of gasoline.” 

9. R : “Do you mean that because of the difference of their initial price?” 

10. A-27 : “Yes.” 


In another case, document on figure 7 shows Rizka (A-17) had activated FMC in his work 
appropriately. Representation capabilities was involved by understanding the presented informationon 
the figure. Then, student activated his reasoning and argumentation by associating information about 
initial and current price for two types of fuel as well as percentage of price increasing concept. In this 
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case, A-17 succeeded in activating receptive communication through comprehending the phrase 
'percentage of price increasing’ well. Those capabilities contributed to the correct formulating process 
he did. A-17 then continued to devise problem solvingstrategybydoing employing process through 
performing symbolic operations to find the percentage of price increasing for each type of fuel 
(premium and diesel) by using simple calculation. The values of percentage for both fuel types he 
obtained in employing process were then compared to draw the final conclusion that the price 
increasing percentage of the diesel is higher than the premium. 


is 0 0 0 „/ 

o 500 - ) ^ ’ 1 * 

L f.W J 

fOM~ 

A-17 

8 500 

£f*L--'If, o(oir 54 , 5*1 Y, 

3000 J 

f 22L 

l ySOO 

Wr-iw-r WflflbuW-rb ([hr Tr f r 

So, the increasing price of diesel and premium 

were different 



Figure 7: Student’s answer (A-17) 


When solving prototype 3 in the field test, in general students considerably performed the low 
achievement in which only 10 out of 28 students whose got score more than 50. However, the 
analysis of students’ answer document and interview result indicates that prototype 3 consisting 10 
PIS A-like taks in this study potentially develop student’s mathematical literacy. It is briefly notable 
that most of students have activated six out of seven FMC (communication, mathematising, 
representation, reasoning and argument, constructing strategies for solving problems, using symbolic, 
formal and operations) into the three mathematical processes (formulating, employing, and 
interpreting) when solving those problems, although those comptences have not maximally activated 
yet. It could not be denied that most of students in this study have not been able to involved the whole 
demanded competences when solving the PISA-like tasks. In other words, most of students merely 
partly-activated FMC within mathematical process in their works. 

Based on the interview result, students admitted that they are not familiar with context-based 
problems like PISA-tasks. It is in line with Ahyan, et al., (2014) who claimed that students are not 
accustomed to solving PISA-like problems in their learning activity at classroom. However, a set of 
PISA-like tasks administered to students in the field test in this study could gave them opportunity to 
emerge, or even to enhance their mathematical literacy. 

Quantitatively, the result also shows that the highest percentage of students’achievement was on 
the interpret tasks (52.55%). While achievement for employ and formulate tasks were 40.74% and 
39.63% respectively. It indicates that most of students could well interpret the results obtained from 
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mathematical calculation they perform by reflecting upon these solutions back to the contect of the 
problem given. 

CONCLUSION AND RECOMMENDATION 

A set of valid and practical PISA-like mathematics problems developed in this study has 
potential effect to develop students’ mathematical literacy, indicated by activation of fundamental 
mathematical capabilities (FMC) that underpins three mathematical processes; formulate, employ, and 
interpret when they solve some PISA-like tasks, although in general these abilities have not maximally 
activated yet. Quantitatively, the result also shows that the highest percentage of students’achievement 
was on the interpret tasks (52.55%). While achievement for employ and formulate tasks were 40.74% 
and 39.63% respectively. Regarding these findings, it is definitely important to prepare students to 
fully grasp the essential ability that every mathematics should have, mathematical literacy, by 
providing them opportunity to promote such FMC within mathematical processess. In this term, PISA- 
like tasks may potentially contribute to the development of this capabilities. 
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